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RANKING  AND  SELECTION  IN  DESIGNED  EXPERIMENTS: 
COMPLETE  FACTORIAL  EXPERIMENTS* 

by 

Edward  J.  Dudewicz 
and 

Baldeo  K.  Taneja 


ABSTRACT 

As  early  as  the  first  paper  of  Bechhofer  on  ranking  and 
selection  in  1954  it  was  recognized  that,  with  a  ranking  and 
selection  goal  as  well  as  with  other  goals  such  as  estima¬ 
tion  or  hypothesis  testing,  it  might  be  desirable  to  carry 
out  one’s  experiment  in  some  design  other  than  the  complete¬ 
ly  randomized  design.  Nevertheless,  over  the  years  almost 
all  of  the  papers  in  the  area  have  developed  their  methods 
and  theory  explicitly  only  for  the  completely  randomized 
design.  In  this  paper  we  review  what  is  known  about  ranking 
and  selection  in  design  settings  other  than  the  completely 
randomized  design,  and  then  proceed  to  new  results  on  the 
complete  factorial  experiment  setting. 


*  This  research  was  supported  by  Office  of  Naval  Research 
Contract  No,  N00014-78-C-0543 . 


1. 


INTRODUCTION 


In  the  first  paper  on  ranking  and  selection,  Bechhofer  (1954) 
recognized  the  importance  of  allowing  for  experiments  carried  out 
in  some  design  other  than  the  completely  randomized  design.  For 
full  references  see  Dudewicz  and  Koo  (1981).  In  this  paper  we: 
briefly  survey  the  state  of  knowledge  of  ranking  and  selection  in 
designed  experiments  (Section  2);  discuss  factorial  experiments 
without  interaction  in  some  detail  (Section  3);  and  give  new 
results  for  factorial  experiments  where  interaction  may  be  present 
(Section  4). 


2.  RANKING  AND  SELECTION  IN  DESIGNED  EXPERIMENTS 


In  the  basic  formulation  of  the  ranking  and  selection  problem, 
there  are  K  populations  (sources  of  observations)  with 

respective  unknown  means  y-p  y  ^  for  their  observations,  normally 

distributed  with  common  known  variance  o2  >  0,  and  the  goal  is  to 
select  a  population  whose  mean  is  V[K]>  where  yj-^-j  5  P[2]  1  ^Tk] 

are  the  ordered  y^,  y^.  Achieving  this  goal  is  called  making  a 

Correct  Selection  (abbreviated  as  CS) .  Bechhofer  (1954)  gave  a 
single-stage  procedure  which  guaranteed  that  P(CS)  >  P*  whenever 


W[K]  “  U[K-1]  -  <5*tl/K  <  P*  <  1,  0  <  <5*). 


There  have  been  many  papers 


written  in  the  area,  with  extensions  in  many  directions;  see  Dudewicz 


(1980). 


Bechhofer  (1954)  dealt  mainly  with  the  completely  randomized 
design,  but  also  considered  the  2 -factor  factorial  experiment  without 
interaction,  giving  the  Factorial  Procedure  SP1  of  Section  3  below. 

He  briefly  noted  how  to  similarly  solve  the  r-factor  factorial 
experiment  (again  without  interaction)  with  SP1.  Also  see  pp.  77-82 
of  Gibbons,  Olkin,  and  Sobel  (1977)  for  examples,  restricted  to  the 
no- interact ion  case. 

Bawa  (1972)  compared  Bechhofer’s  SP1  to  the  traditional  One-at- 
a-time  Procedure  SP2  in  a  no-interaction  setting,  and  found  SPl  is 
superior  in  asymptotic  efficiency. 
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A  procedure  for  the  factorial  experiment  with  interaction 
(Interaction  Procedure  SP3  in  Section  3  below)  was  given  in 
Dudewicz  (1977),  and  independently  simultaneously  by  Bechhofer 
(1977). 

For  the  problem  of  selecting  the  best  regression,  Dudewicz 
(1976,  Section  14.4)  noted  the  solution  (also  see  Chanter  9  of 
Gibbons,  Olkin,  and  Sobel  (1977),  with  an  erroneous  claim  of 
originality  on  p.  241).  The  problem  of  selecting  the  largest 
interaction  was  addressed  by  Bechhofer,  Santner ,  and  Turnbull 
(1977).  Rinott  and  Santner  (1977)  applied  inequalities  to  design 
an  experiment  to  select  the  best  treatment  in  an  analysis  of 
covariance  model. 


3.  RANKING  AND  SELECTION  IN  FACTORIAL  EXPERIMENTS: 

NO  INTERACTION 

Suppose  an  r-f actor  (r  >_  2)  factorial  experiment  with  k.. 

(kj  i  2)  levels  of  factor  j  (j  =  1,  2,  ...,  r).  Assume  observations 
are  normally  distributed  with  known  common  variance  o2  >  0  and  that 
for  an  observation  taken  at  (level  i^  of  factor  1,  level  i2  of  factor 
2,  ...,  level  i  of  factor  r) 


<i) 


E(X  .  . ..  j  )  =  M  +  I  a1 

1112  r  j=l 


(i-  =  1,  2,  ...»  k.  *,  j  =  1,  2,  ...,  r)  where  p  and  a-J  are  unknown 
3  1 

parameters  with 

k. 

J 


l  *  =  0  ( j  =  1 ,  2  ,  .  . .  ,  r) . 


i4  =  l  3 

j 


For  each  fixed  j  (j  =  1,  2,  r) ,  let 


0<3)  <  a(j)  < 
“[1]  -  “[2]  -  * 


<  > 
-  °[k.] 


3 


denote  the 


in  numerical  order.  Each  factor-level 


combination  (i^,  i2 »  •  ir)  is  called  a  population ,  and  a  "best" 

population  (among  all  K  =  k^k2  •**  ^r  Populations)  is  any  one  with 

maximum  E(X-  .  . ..  •  ),  i.e.  any  one  with  mean 

1-112  *r 

y[K]  =  u  +  a[kJ]  +  a[k2]  +  +  a[J^]  • 

Define  (for  j  =  1,  2,  r) 

6j  =  a[k3>]  ~  a[k?-l]' 

Then  set  6*  =  (6^,  6/,  5r*)  and  P*  (0  <  6^,  6 2*,  6 r*  < 

1/K  <  P*  <  1)  and  seek  procedures  6>  which  guarantee  the  probabilify 
requirement 

P<CS j(P  >  >  P*  whenever  &  >_  Q*  componentwise. 


Three  procedures  proposed  in  the  literature  are  the  following. 

Procedure  SP1  (Factorial  Procedure).  Take  N  independent 
observations  from  each  of  the  K  populations.  Let  denote 

the  (marginal)  sample  mean  of  all  observations  having  the  jth 

factor  at  level _i.  (j  =  1,  2,  ...,  r).  For  each  j,  let 

*l\]  <  •••  _<  denote  ^  ,  ...»  ^  in  numerical  order. 

3  3 

Select  the  level  associated  with  the  largest  (marginal)  sample 

mean  of  each  factor,  i.e.  with  7^^,  ...,  and 

1  2  r 

assert  that  that  population  (factor  combination)  is  best.  (NK 
observations  are  used.) 

Procedure  SP2  (One-at-a-time  Procedure).  Fix  the  level  of 
each  factor  except  factor  j ,  take  hj  observations  at  each  level 
of  factor  j ,  and  compute  the  sample  mean  at  each  such  level. 
Then  select  the  level  (of  factor  j)  yielding  the  largest  such 
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1 


t  *  y 


sample  mean.  Proceed  similarly  for  j  =  1*  ...»  r  (keeping 

"selected"  levels  fixed  for  the  corresponding  factors),  taking 

a  set  of  independent  observations  at  each  stage  of  experimentation. 

finally  state  that  the  population  corresponding  to  the  selected 

factor  levels  is  best.  (  T  k.  N.  observations  are  used.) 

-  3  3 


3  =  1 


Procedure  SP3  (Interaction  Procedure).  Take  M  independent 
observations  from  each  of  the  K  populations.  Select  the  population 
yielding  the  largest  sample  mean  as  best.  (MK  observations  are 
used .  ) 

In  order  to  compare  procedures  SP1,  SP2 ,  SP3  let  n1  ,  n2  ,  n^ 
denote  the  respective  smallest  number  of  observations  each  procedure 
needs  in  order  to  guarantee  the  probability  requirement .  From  Bawa 
(1972)  we  know 


n,  =  min  (NK:  N  such  that  H  (1-e  ^  -*)>_?*} 

3  =  1 


and 

r  r  -a-N. 

n0  =  mini  l  k-N.  :  (N.  ,  N)  such  that  H  (1-e  3  -1  )  >  P*l, 

2  3=i  3  1  1  r  3=1 

where  (for  j  =  1,  2,  ...»  r) 


(S*)2 


3  3  = 


Nr  = 


4o  ' 


NK 

*7 


Using  asymptotic  calculus  as  in  Bawa  (1972)  and  Dudewicz  (1969) 
it  follows  that 


K' 

n.  -v  —  4oJlog(  1-P* )  max  { - J — }  +  o(l-P*). 

1  l<j<r  (Sp2 

r  k. 

r._  -v  -4o 2 log(  1-P*)  l  - JL—  +  B 

2  5=i  cs»)! 

where  a  ^  b  means  the  limit  of  a/b  is  1  as  P*  +  1  and  B  does  not 
depend  on  P*.  Also  (Dudewicz  (1969)) 


where  P(CSjSP3)  >  P*  is  guaranteed  for  6  =  -  H[k-i]  -  ** • 


Therefore 


k. 

max  ( - - - ) 

n,  l<j^r  (6-*) 2 

ARECSP1,  SP2  )  =  lim  — =■  =  ™ - c — J -  » 

P*-l  n2  |  3 

3  =  1  C  6  j  * ) 2 

nl  (6*>2  t  i 

ARL(SP1,  SP3)  =  lim  =  ■■  ■-  -  max  { - "2— } » 

P*^l  n3  K  l<j<r  ( 6  -  * ) 

n2  (6*)2  r  ki 

ARE ( SP2  .  SP3 )  =  lim  -=-  =  -  -i, —  )  - J • 


ARE(SP2,  SP3)  =  lim 

P*+l  n  3 


j  =  l  (S.*)2 


Note  tnat 


a  ■  «[K]-“CK-1]=  <"♦  i^tkV  -  £fir  *  X  “&■:  •  6i’ 


=  mini^  ,  .  .  .  ,  6r) , 

hence  6  >  6*  and  6*  =  min(51*,  <5r*)  furnish  comparable  probability 

requirements  for  SP1 ,  SP2 ,  and  SP3.  Letting  c  be  that  j  (1  f.  j  £. 
where  k^/CS^*)2  is  maximized, 

2  3c 

ARECSP1 ,  SP3)  =  - #  '  1 

(6c.)>  *  - 


(since  <5*  <  6C*  and  kc  £  K),  and 

( 5* ) 2  r  kc 

ARE(SP2,  SP3 )  <  •2~2— - r~  <  1  . 

-  K  - 

Hence  SP1  and  SP2  are  each  more  efficient  than  $P3.  In  the  "symmetric' 
case  =  k2  =  **'  =  kr  =  k  and  5l*  =  52*  =  •••  =  5r*  = 

ARECSP1,  SP2)  =  i,  ARE(SP1,  SP3)  =  -~T  ’ 
r  k 

ARECSP2 ,  SP3 )  =  — . 


6 


I 

I 

I 

I 


t 

ii  > 


Thus,  when  there  is  no  interaction,  SP1  is  best  and 
SP3  is  worst.  However,  we  see  (Section  4)  that 
SP3  is  fully  resistant  to  interaction,  whereas  SP1  and  SP2 
are  not . 

4.  RANKING  AND  SELECTION  IN  FACTORIAL  EXPERIMENTS;  INTERACTION 

We  will  now  study  factorial  exneriments  with  interactions. 
So,  we  suppose  an  r-f actor  (r  >  2)  factorial  experiment 
with  k.  levels  of  factor  j  (j  =  1,  , . . ,  r)  and  assume  our 
observations  are  normally  distributed  with  known  common 

variance  a2  >  0  and  that  if  X-  •  •  is  an  observation  taken 

1112‘ • ulr 

at  [level  i^  of  factor  1,  level  i2  of  factor  2,  ...,  level  ir 
of  factor  r}  then 


E(X 


<il) 


(iii2} 


i  i  i  >  =  VI  +  I  ♦  l  l  a./.;  ♦  . 

ll12 ’ * ' xr  jx=l  l3j.  32=1  3!=!  32 


A.  ,  A  r;L  ai,  i,  i 


^r-l1  -^r-21  31=1 


’1  J  2 ' 


r-1'  (12. ..r) 

+  a . 

1112 ’ ' ‘ 1r 


3r-l 


(ij  =  1,  2,  ...,  kj  for  j  =  1,  2,  r)  where  »  and  the 

a*s  are  unknown  parameters  with 


31 

l 


t3n> 


tti  ~  0 

V1 


for  jx  =  1,  2, 


k5, 

/  'J1J  2 
.1  ai.  i 
M  =1  3n  3 
31 


(3132} 
llj2 


3  2 
l 


(3132)  .  0 
.  ,  ai.  i.  -  0 

^  =1  3  n  3  - 


3. 


*1  J  2 


(for  j,(=l,  2,  r)  <  j?C=l,  2,  ...»  r))i 


7 


>  <W3>  =  "I2  J3  a''1!23!’  '  0 

^  ,  ai.  i4  i-;  ,•  M  i.  =1  13-,1jo  3i 


V1  V^s  V1  jl  j2  ^3 


^3=1 


3 1  3  2  ^  3 


(for  ^{=1,  2,  ...»  r)<12C  =  l,  2,  r)<j3(  =  l,  2,  ....  D); 

and  so  on;  and  lastly 


kl 

y 

L 

i  1=  1 

(1  2 . . .r) 
a  *  * 

ili2  * ’ *  r 

k2 

=  V  a^1 . 1  *  •  *1f)  =  ...  = 

i^=l  i1i2,*'1r 

kr 

l 

V1 

a(l_2...r)  __  0 

1 . 1 « • • • 
i.  Z  *■ 

U3 

=  r,  2 ,  . . . , 

k.  tor  3  =  1,2,  . ..  ,  r). 

For 

each  fixed 

j  <3 

=  1,  2,  . . . 

,  r ) ,  let 

a[i]  i  “cii  -  ••  • 1  at*-i 

denote  the  a<j\  a<j),  ...,  ^  in  numerical  order. 

3 

This  r-factor  experiment  may  be  conceived  of  as  an  r- 
dimensional  cuboid.  Each  factor-level  combination 
(i  ,  i  ,  . . . ,  i  )  is  called  a  population,  and  a  "best” 
population  (among  all  K  =  k^...^  populations)  is  any  one 
with  E(X.  .  •  )  maximised  over  all  r-tuples  (i^>  *2’ 

ip)  tcalled  cells) . 

Once  observations  have  been  taken,  let 

denote  the  sample  mean  of  all  observations  m  cell 

(i  ,  i  ,  i  )»  and  define  the  (marginal)  sample  means 

X  *  2  ^ 


v 


k2  k3 

l  *  ’ 

i2=l  i3-l 


kr 

1  ki  i 

v-1  1  2' 


^2^2  *  *  * 


(i^  -  1,  2,  ...»  k^) 


3 


•  • 


•  1 2  •  *  *  •  ♦ 


.1  . 1 

1^=1  13s! 


k 

f  *■ 

xr_1 


1li2 


.  x. 


klk3 


.  K 


Ci,  -  1 5  2 )  • • • ,  kn)j 


5f... 


r  f  . 

xl'1  x2  =  1 


k^'k^'. 


>-l 

.  I  7, 

1r-l=i 

7T 


xlx2 *  *  *  xr 


kr-l 


( =  1,  2,  ....  k,,) 


We  now  wish  to  prove  (as  indicated  briefly  in  Section  3)  that 
SP1  and  SP2  are  vitiated  by  interaction,  while  SP3*s  validity 
remains  unaltered. 

Theorem  1.  Let  n^  be  the  smallest  sample  size  for  which 
procedure  SP1  guarantees  F’(CS|SP1)  >_  P*  whenever  £  ^  in 
a  model  with  zero  interactions.  Then  in  a  general  (non¬ 
zero  interaction)  model, 

inf  P(CS)SPl)  <  y 
fl(6*) 

where  Q(5*)  =  {(p^  P2>  •••»  :  w[K]"P[K-l3  -  6*’ 

6*  =  min{6^*,  ...» 

Before  proceeding  to  prove  Theorem  1,  we  will  show 
that  (in  the  presence  of  interaction)  not  only  may  the  cell 
selected  by  SP1  not  be  the  best  cell,  but  it  may  be  the 
case  (with  high  probability)  that,  for  each  of  the  r 
factors,  no  column  with  the  respective  largest  (marginal) 
sample  mean  contains  the  best  cell.  The  proof  will  utilize 
an  addition  of  interaction  to  achieve  such  a  situation. 

To  verify  the  claims  of  the  above  paragraph,  consider 
a  model  with  no  interactions.  By  labeling  and  relabeling 
the  cells ,  we  may  arrange  our  experiment  so  that 


9 


£<X. 


1112* ' * 1r 


Cl) 

)  -  u  +  ar . 


(2) 


ti13  +  aCi2]  + 


(r) 

*  +  “Ci  ] 
r 


Cl-.  -  1  ?  2  ^  •  •  *  t  ^  *  i  o  ~  i  >  2  ^  ^  1c  5  •  *  .  j  i  5  2  ,  •  *  •  ^ 


J1  C  3 1 ) 

still  with  l  a.  =0  for  =  1,  2,  . . . ,  r.  (Thus, 

i.  =1  xj.  x 

3i  1 


cell  Cl,  1,  ....  1)  is  now  the  worst  cell,  while  cell 
(k^,  kj ,  kr )  is  the  best.) 

Now,  to  achieve  the  desired  situation,  we  assume  without 
loss  of  generality  that  >_  2  (j  =  2,  3,  .  .  .  ,  r)  and 

that  all  interactions  are  zero  except  for  highest-order 


interactions.  For  the  case  ki  =  2,  see  Remark  2  following 

the  proof  of  Theorem  1.  In  the  case  >  2 ,  we  add  highest- 

order  interactions  to  each  E(X.  .  ...<  )  as  multiples  of  a 

111 2  r 

number  "a"  (yet  to  be  chosen)  according  to  a  scheme  such  that 
cell  (I,  1,  1)  becomes  the  best  cell  for  large  positive 

"a".  To  motivate  this  scheme  for  r-factor  experiments,  we 
first  illustrate  it  for  two-factor  experiments,  next 
show  how  to  extend  it  to  three-factor  experiments,  and 
finally  explain  the  general  case  (r  >  3). 

For  a  two-factor  experiment  (r  =  2),  add  multiples  of 

a  (yet  to  be  chosen)  to  each  E(X.  .  )  as  specified  in 

1112 

Figure  1  to  form  model  E(X.  .  (a)).  Note  that  there  is  a 

ili2 

simple  pattern  over  most  of  the  table  of  Figure  1  (except 
for  row  1  and  column  1)  and  that  (except  for  cell  (1,1)) 
row  1  and  column  1  entries  are  chosen  so  the  appropriate 
row  or  column  sum  is  zero.  Finally,  the  cell  (1,1)  entry 
is  chosen  so  that  the  row  1  and  column  1  sums  are  each 
zero  also.  The  added  multiples  of  "a"  are  the  interactions 

(a),  and  satisfy  J  (a)  =  l  (a)  =  0 

1112  i  _i  1i12  i  _i  J'i12 
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for  all  a.  (Note  that  cell  (1,1)  is  best  for  large  positive 

a. ) 

How  "a"  can  be  chosen  so  that,  letting  f  •  •  (k-=  ,k«)  be 

12 

the  Figure  1  entry  in  row  and  column  i2,  we 
simultaneously  satisfy 

vi+otcij+otCli+aii2  5  (^^vi+otCi^]+cxCi2  3+otiii2Ca)  +  6*<VCil>i2>??  (1  ,i:>  5  ’ 

i .  e . 

"*“a:+“cu*fn(ka>k2)ai‘,,'a[i’:*“U2ffi1i2(ki’k2)a+6“(v<ii'i2>l'(1>11) 


‘“u,1]  ♦  »u’]>  -  <“m  *  «{!!>  *  s’ 

flltkl'k2)-fi1i2tkl’k2^ 


(V(i1,i2)  i  (1,1)) 


(where  it  is  evident  from  Figure  1  that  the  denominator  is  >  0) 
For  this  purpose  it  suffices  to  take 


a  =  6 


*  -R’i  -  {°ci]  ^  "m1  *  “ 


:1-l)(k1-2)(k1+3) 


For  a  three-factor  experiment  (r  =  3), 

(i^  =  1,  2,  ...»  k-^j  i2  =  1,  2,  k2;  13  -  1>  2, 

where  y  and  ct's  are  unknown  parameters  with 


,  kq  )  5 


l  =  I  «|2>  «  .1 

i  =1  11  i„ = 1  x2  i„= 1 


,  (3 )  . 


=  0, 


k,  ko 

r1  (13)_  .(13). 


k?  k3 

r  C23)_  r 


d  aU2)  =  l‘  a<12>=  f  .‘“>  =  l  a^1--  l  “F'iJ=  I  “i2?5"0* 

4  =  1  ll*2  4  =  1  1112  4  =  1  1113  ij  =  l  13  4=1  1213  i3=l  l2l3 
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0. 


!! 


a'1?3? 

Xl1213 


Figure  2.  Three-factor  experiment  in  the  form  of  a  cuboid. 


Figure  2  shows  the  three -factor  exepriment  as  a  cuboid. 
We  now  add  second-order  interactions  to  the  cells  as 
multiples  of  a  (yet  to  be  chosen),  as  follows.  In  the  last 
k^-1  of  the  k-,  levels  of  factor  3  (i.e.  levels  ..., 

kg  of  factor  3)  add  -f^  ^  (k-^kjJa  in  cell  (i^,  i2>  13)* 

13 


I  !  2 


f 


■4 


where  f.  .  (k-.  ,  k,)  is  as  specified  in  Figure  1.  The 
il12  z 

other  levels  of  factor  3  have  zero  second-order  interactions 
in  each  cell,  except  for  the  first  level,  in  which  we  add 
f i  j  (k-^,  k2)a(k-L-l)  in  cells  (i^,  i2 ,  1),  where  again 

f.  •  (.k,  ,  k0)  is  as  specified  in  Figure  1.  This  makes 
i1i2  i  2 

the  column  sums  Ccorresponding  to  factor  3)  zero,  and  the 

interactions  all  satisfy  the  model  restrictions,  (Note 

that  cell  (1,  1,  1)  is  best  for  large  positive  a,) 

Let  f  -  .  .  Ck,  ,  k„ ,  k.,)  denote  the  number  of  multiples 
12  3  1  1  3 

of  a  added  to  cell  (i^,  i2,  i3)  Ci-^  =  1,  k-^*, 

i2  =  1,  k2;  i3  =  1,  k3).  Then  a  can  be  chosen 

so  that  we  simultaneously  satisfy 

^+“ci]+aCl]+a[l]  +  flll(kl’k2>k3)a^^aCiJ]+  a[i*]  +  4V31 
+  fi1i,i  <k1,k2,k3)a  +  6‘;'(V(i1  ,i2  ,i3)  /  (i,  i,  D), 

-i  t  j 


f«(1)  +«<2)  +«(3)  >  .  (3)>+.* 

'“ti,  ]  aCi?]  a[i,]}"{0t[l]+oCl]  °[1]}  4 

^  >  - r 7~r — ~z - s - - -  CV(i  i  i  )*(l,l,l)) 

f 111 (kl  ,k2  ,k3 }  "  fi1i2i3^i»k2,k3)  1  2  3 

(where  the  denominator  is  >  0). 

For  this  purpose  it  suffices  to  take 

{4l[l  +  “Ck2J  +  “[k3]J-t“U]  +  “[1]  +  + 


(k1-2)(k1-l)k1(k1+2) 

The  general  case  of  an  r-factor  experiment  (r  >_  2)  is 
handled  by  induction  on  r,  using  the  method  shown  to  proceed 
from  r  =  2  to  r  =  3  above.  Namely,  suppose  we  have  an 
interaction  scheme  for  an  (r-1) -factor  experiment  where 
all  the  interactions  are  zero  except  the  highest-order 
ones.  In  order  to  extend  it  to  an  r-factor  experiment, 
suppose  that  the  r£h  factor  has  kr  levels.  In  the  last 
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(kl~l|  l0vels  of  factor  r  (that  is,  from  k^-J^  +  2  through  kp , 
both  inclusive),  we  add  interactions  of  highest-order  as  in 
the  (r-1) -factor  experiment  scheme,  but  with  sign  of  each 
entry  reversed.  The  rest  of  the  levels  (except  for  the  first 
one)  receive  zero  highest-order  interaction  in  each  cell. 

Then  to  make  appropriate  interaction  sums  zero,  we  add 
highest-order  interactions  in  the  first  level  in  such  a  way 
that  each  entry  is  (k^l)  times  the  corresponding  entry  of 
the  (r-1) -factor  experiment  scheme.  As  before  we  may 


find  a  large  positive  number  a  such  that  cell  (1.  1.  ...,  1) 
is  best. 

We  are  now  in  a  position  to  prove  Theorem  1. 

? Poot  of.  Theorem  1.  Let  for  SP1  satisfy  PCCS|SP1)  >  P* 

whenever  £  >_  in  a  model  with  zero  interactions  (which,  of 

course,  implies  a  sample  size  N  per  cell,  with  n,  =  NK). 

Now  5T  y  c  * 

1 . a2 . 1  •••»  >  are  independent  normal 

random  variables  with  (possibly  different)  means  and  the 
same  variance  k-jO  2 / ( NK ) ,  hence 


V  * 

’  *2.. 


»  * 


are  independent  N(0,1)  random  variables  when  we  define 


Now  consider  model  E(X.  .  (a))  with  "a**  chosen  so  large 

positive  that  cell  Cl,  1,  ...,  1)  is  best,  according  to 
the  interaction  scheme  previously  given.  Then,  in  this  model 
with  interaction. 


15 


P(CSjSPl)  = 

=  max(Jr, 

v 

y 

> 

•  •  •  > 

*1, 

=  max(Sf 

J7 

•  >  A  .  ?  # 

y 

... ,  7 

• 

_  1  =  max  (5?  _ 

*  .  . 

,  2  * 

. . .  ,  5< 

= 

p[yn 

>  ^  >  Yi2  >  ^  »  •  • 

>  0  ; 

Y21 
*  •  • 

>  ® *  Y22  >  0  ’  '  * 

*’  Y2»k2-1 

>  0; 

Yrl 

>  0,  Yr2  >  0,  .. 

•*  Yr,kr-1 

>  03 

where 

yh  *  *i.„ 

- 

Xi  +1  .  =  >  2 , 

•  ■  ■  > 

kx~l ) 

Y2i  ‘  *.l., 

.i.+i  _ _  ^9  =  1 »  2  > 

*  '  M 

k2-l) 

’  2  . 

Yri  =  * 

*  Ar  •  •  *  ’ 

.  ..1  ' 

* . iTn  (ir=1’2- 

'  '  '  5 

kr"1)' 

>; 

>3 


N°w  Y2i>-*->  Y2,k2-1;  Yrl’’**> 

has  a  (k-^  +  k2  +  .  .  .  +  kr-r) -variate  normal  distribution  with 

E(Y  \  -  a(D  ) 

M  li,'  -  a[i,]  -  a[i,+i 


(2)  (2) 
[Ci2]  *  a[i2^ 


Var(Y  .  )  = 
1 

4  V  A. 

NK 

(il  * 

2o2k0 

Var(Y~ •  )  = 
2 

-rnr2 

(i2  = 

E(Y  .  )  = 
rir 

a(r) 

“Cir] 

„(r) 

-  aCir+l3* 

Var( Y  .  ) 
rir 

a2k1 

CovCY  -  , 
1 

Yiq> 

=  ~W~  Cll 

*  i£). 

ozk2 

Cov(Y„.  , 
iX2 

Y2i'> 

-  "N ir  (l2 

*  ip. 

2o*k_ 


NK  r 


(  iy,  *■  1  s  2  ,  ■  • .  j  k^“l )  > 
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2  >C_ 


C°vtYri  •  Yri'>  ■  TST  Cir  "  K>' 
r  r 

Cov^ji.’  Y£i  )  =  0  (j*  *»  j  =  1»  ...i  r*?  £  =  1,  r) 

3  ^ 

*  i  *  2  , .  ,  .  ,  k  x  “1 ,  ^2  ~  1  ?  2)1*1)  k2"l )  ■  ■  *  )  3-^  *"  3.  j  2  j  *  *  *  j  k^-*  1. ) 


Thus  <Yi;l,.  .  .  ,Vljk^_1)  »<-Y21,,  .  .  ,Y2?k  _x)  .  .  ,CYrl>.  .  .  ,Yrik  _x) 

independent ,  so 


are 


P(CS|SP1)  =  J[  P[Yn  >  0,  Y*  2  >  0,  ...,  Y.  ,  ,  >  0], 

5  =  1  -1  ■*  j5K-i~ 


Considering  the  first  factor, 


PtYl1>0*  Yi?>0’- • • >  Y 


12' 


>  0] 


=  PCX, 


=  max 


<x. 


<  PCJfJ 


=  maxCX* 


»X, 


>  •  •  • 


>3 


T 7— 
K1 


Proceeding  similarly,  we  find  that 
32 


PC Y^ 1  >  0,  Y.p  >  0,  Y.  k  ,  >  0]  <  -£ 

3  ’  3 


hence 

P(CSjSPl)  <  1/K 
as  was  to  be  proven. 


(j  s  2 ,  .  .  .  ,  r). 


3 


Remark  1 .  Note  that  for  all  schemes  of  adding  interactions 
and  for  all  >_  2  (j  =  1,  2,  ...,  r)  ,  procedure  SP1  still 
achieves  probability  at  least  P*  for  selecting  the  cell 

,<3> 

C3 


associated  with  u  +  £  aripT  ( which  may  not  be  the  "best" 

5  =  1  LV 
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cell  after  adding  interactions).  This  is  true  because 
the  sum  of  interactions  in  each  row  and  in  each  column 
is  zero  and  so  the  (marginal)  sample  means  do  not  change. 


Remark  2 .  In  the  case  k-^  =  2 ,  we  add  interactions  in 
such  a  way  that  cell  (k^,  ,  .  ..,  kp)  does  not  remain 

best.  From  Remark  1,  it  then  follows  that 

P(CS|SP1)  <  1-P*  . 

Theorem  2 .  Let  t\2  be  the  smallest  sample  size  for  which 
procedure  SP2  guarantees  P(CS|SP2)  >  P*  whenever  6  >  6* 

.  —  %  —  'Xj 

m  a  model  with  zero  interactions.  Then  in  a  general 
(non-zero  interaction)  model, 

inf  PCCS | SP2 )  <  1 

^  ( <5  * )  min  kT 

l<j<r  -1 

wheie  (2(6*)  -  { (p^  > .  .  .  ,  UK)  :  Pj-  K]-p  ]><$*  » 6*=min{  6  J  , .  .  .  ,6 

Proof  of  Theorem  2 .  Let  n^  for  SP2  satisfy  P(CSjSP2)  >  P* 
whenever  6  >_  6*  in  a  model  with  zero  interactions.  Let  factors 
be  ordered  as  in  the  discussion  following  the  statement  of 

Theorem  1,  and  consider  model  E(X.  .  .  (a))  with  a  set  so 

1l12,,,1r 

that  cell  (1,  1,  ...,  1)  is  best.  Suppose,  without  loss  of 
generality,  that  the  experimenter  decides  to  first  fix  levels 
of  each  factor  except  factor  j  (and  hence  experiments  across 
the  levels  of  factor  j).  Define  events 

F..  =  {Experimenter  starts  with  level  i.  of  factor  i) 

j  1 

Eji.  =  {ExPerimenter  selects  level  ij  of  factor  j), 

(ij  =  1,  ...»  k^;  j  =  1,  ...»  r)  so  that  event  E11E21*-.E„1 
corresponds  to  a  correct  selection.  Now,  for  any  e>0,  a  may 
be  taken  sufficiently  large  that 
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•Kb 


P(E 


•  •  •  |  F i  ^  )  £  £ 

j 


11  21  '  •  *  “rl'*3 


(ij  *  1). 


Now,  letting  F.  =  F . ,  U  ,  .UF-,  (the  event  that  the 
1  It  ]K; 


experimenter  starts  with  factor  j)  and  taking 


P(F.,  | F. )  =  l/k4  as  a  reasonable  model 

1 K ,  3  J 


P<Fjl|Fj)  =  ... 

in  light  of  no  prior  knowledge  (on  the  part  of  the  experimente 


as  to  which  levels  are  better  than  others,  we  find 


kj 


P(CS  j  SP2 ,  F.)  =  J  P(E  Z2,  ...  Erl  E.ijF.) 

J  -»  J 


V1 


f  =  1  P<Fji^F3)P(Ell  E21  ErllFjij 


k  . 

P<Fj 2. !  Fj  >PCE11E21  ’  *  *Erl!Fjl)  +  .£ 

k.  ^  = 

<  itr*  f  +  z, 

1  i j  ~  2  kj  kj 

and  the  theorem  follows. 

Theorem  3.  Let  n^  be  the  smallest  sample  size  for  which 
procedure  SP3  guarantees  P(CS|SP3)  >  P*  whenever  6  >  6*  in 
a  model  with  zero  interactions.  Then  in  a  general  (non¬ 
zero  interaction)  model, 

P(CS | SP3 )  >  P*  whenever  P[K]  -  P[k-i]  t  =  min { 6* , . , . , 6* ) . 


2  P(F3i, 


lFj>P<E11E21- 


-Erl|F 
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Proof  of  Theorem  3 .  In  a  model  with  zero  interactions,  suppose  that  cell 

(m^,  m2 ,  mr)  is  best.  Then,  if  M  observations  per  cell  have  been  taken, 

whenever  6  >  6*  componentwise  we  have 
\ 

PCCSI SP3  >  -  P[5T  =  max  (X-  .  .  )  , 

i,  =  1 ,  2  ,  . . .  ,  k.  ( j  =  1  ,  2  ,  . . .  ,  r) ] 

L t x  „  )-r(x,  ,  •  ) 

m  -  m  - .  .  .  tn 

=  P[Xf  ..  .  <  X*  - ^ - - - — - L-  , 

1  ‘  1  -  fT7 


1 1  -  1 ,  ^  ,  •  •  -  ,  k  •  ( j  =  1 ,  2  ,  .  •  .  ,  r )  ] 


>  P[X*  .  .  <  X* 

1l12',,1r  mlm2 ' • ’ mr 


min {6$ ,...,5*1 


i- =1,  2 ,  . . .  ,  k-  { j  =  1 ,  2 ,  .  .  . ,  r) ] 


v  i  *4!  mini  , .  .  .  ) 

inf  P(CSjSPl)  r  /  ^<x  +  - - - - 

S  >  5  * 
x—% 


)  $ (X)  dX  , 


where  <f  ( •  )  and  ${  * )  denote  the  standard  normal  dist  ri button  and  density 
functions,  respectively.  For  this  inf  to  be  >  P*  we  need 

hr (P*)o2 
v.  ^  n 


—  Milan  i  u  j  ,  .  .  .  ,  u-  j  r 

where  h^tP*)  is  the  solution  h  of  the  equation 

/  ❖K'‘1(x+h)t)i(x)dx  =  F*  . 


Now  in  a  general  (non-zero  interaction)  model  suppose  that  cell 
(c,,  c,,  ...,  c,.,)  is  best,  M  as  above  is  used,  and 
r  kM’i'f  x  it  >  <5*  -  mini  6(* ,  ...,  <$*}.  Then 

LkJ  Lk-U  —  l  -  r  f  v  )-E(X-  •  ' 


PCCSISP3)  --  P!  X*  .  .  <  X* 

1  1 , 1  ~  .  1  —  C 


cle2***cr 


1 11 2 '  -  '  1r 


i.  =  l,  2,  .  .  .  ,  k.  C  3  =  1  ,  ? . r)] 

J  J 

>  P[JT*  .  <  X*  +  , 

-  1.1-...1  -  C.Ct-.C  o  ’ 

12  r  12  r 


ij  =  1,  2,  k^(i  -  1,  2,  ...,  r)] 

=  /  $K-1(x  +  )^{x)dx  >  P* 

L  a  — 


and  the  theorem  follows. 
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Thus,  while  procedure  SP3  is  fully  robust  to  the  pre¬ 
sence  of  interaction,  procedures  SP1  and  SP2  may  be  fully 
vitiated  by  interaction.  One  may  therefore  wish  to  estimate 
interaction  size  and  choose  accordingly  between  SP1  and  SP3 
in  practice.  An  SP4  which  incorporates  this  idea  (acting 
as  does  SP1  when  interactions  are  "negligible"  and  as  does 
SP3  when  interactions  are  "large")  is  now  being  developed. 
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